Systems and Methods for 4-D Hyperspectral Imaging

ABSTRACT

Systems and methods for hyperspectral imaging are described. In one implementation, a hyperspectral imaging system includes a sample holder configured to hold a sample, an illumination system, and a detection system. The illumination system includes a light source configured to emit excitation light having one or more wavelengths, and a first set of optical elements that include a first spatial light modulator (SLM), at least one lens, and at least one dispersive element. The illumination system is configured to structure the excitation light into a predetermined two-dimensional pattern at a conjugate plane of a focal plane in the sample, spectrally disperse the structured excitation light in a first lateral direction, and illuminate the sample in an excitation pattern with the one or more wavelengths dispersed in the first lateral direction.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of and incorporates by reference thecontent of U.S. Provisional Pat. App. No. 62/342,252, filed May 27,2016.

BACKGROUND Technical Field

The present disclosure generally relates to the field of imaging and tomicroscopy systems and methods. More particularly, and withoutlimitation, the disclosed embodiments relate to systems and methods forhyperspectral imaging through the use of spatial light modulators anddispersive elements.

Background Description

Hyperspectral imaging is a remote-sensing spectroscopic method typicallyused in non-medical applications, such as in material identification,astronomy, surveillance, and geophysical applications. It is a method of“imaging spectroscopy” that combines the spatial resolution of imagingwith the chemical specificity of spectroscopy. Light collected from anobject is dispersed into a spectrum of wavelengths or narrow spectralbands, and detected on a two-dimensional (2-D) imaging sensor as a setof images, each image representing one of the wavelengths or spectralbands. Therefore, datasets collected by hyperspectral imaging systemsare often referred to as a hyperspectral cube, represented inthree-dimensions (3-D): two spatial directions (x direction and ydirection) and one spectral dimension (A).

Hyperspectral imaging is an emerging technology for medicalapplications, such as in disease diagnosis and surgery. Biologicaltissues have intrinsic and extrinsic optical signatures, such asendogenous fluorescence and exogenous fluorescence, that can reflecttheir chemical, biophysical, and/or morphological characteristics.Hyperspectral imaging can be applied to investigate the physiologicand/or pathologic changes in living tissue or tissue slices and furtherprovide information about the health or disease of the tissue. Forexample, hyperspectral imaging may replace biopsy as a digital pathologytool. Typically, frozen-section biopsy is used for obtaining informationabout a tissue sample for an oncology surgeon to make decisions duringsurgery. Such biopsies may take about 10 to 20 minutes and employ quickstaining protocols that render poorer feature definition than standardstaining methods. By imaging autofluorescence from the tissue sample andresolving the spectral signatures of the autofluorescence, hyperspectralimaging may be applied as a more rapid histopathology analysis tool thatcould allow for higher diagnostic accuracy over frozen-section biopsies.Additionally, hyperspectral imaging may be applied as an experimentaltool for research and clinical studies, such as applications inimmunohistochemistry (IHC) staining and fluorescence in situhybridization (FISH), in which molecules labeled with variousfluorophores with different spectral signatures are targeted to specificproteins and nucleic acids.

The information in the spectral dimension of a hyperspectral cubetypically reflects the light intensity over a range of wavelengthsemitted by fluorophores or other types of optical labels when they areexcited at a given wavelength. However, if the light intensity ismeasured as a function of both the excitation wavelength and theemission wavelength, more precise identification of the opticalcharacteristics of the fluorophores, optical labels, fluorophore-taggedmolecules, and/or the biological tissue is possible. In addition, inmany medical applications, various types of intrinsic or extrinsiclabels having different spectral signatures may be used, includingfluorophores, Raman labels, photoluminescence labels, or quantum dots(QD). Measuring the combined excitation and emission spectra may enhancethe ability to distinguish, identify, and characterize different labelsof a given type or various types of labels.

Acquiring hyperspectral imaging datasets having both excitation andemission spectra would normally be extremely time-consuming because itwould be necessary to measure the intensity of emitted light at multipleexcitation and emission wavelengths. For example, many widefield 2-Dfluorescent images can be acquired with different filters on acollection path, each filter transmitting a narrow spectral band in theemission spectrum. To acquire the additional excitation spectrum, thisprocedure then needs to be repeated at different excitation wavelengthsusing many different excitation lasers. Furthermore, most of thecollected fluorescence light is discarded if narrowband filters are usedone at a time. Thus, such a procedure is inefficient and could evenresult in photobleaching of the fluorophores in the sample such thatthey are permanently unable to fluoresce. Therefore, there is a need forrapid, efficient, and automated methods and systems for acquiringhyperspectral imaging datasets with both excitation and emissionspectra.

SUMMARY

The embodiments of the present disclosure include systems and methodsfor achieving hyperspectral imaging that allows for acquiring ahyperspectral-imaging dataset with both excitation and emission spectra.Advantageously, the exemplary embodiments allow for rapid, efficient,and automated acquisition of a four-dimensional (4-D)hyperspectral-imaging dataset, including two spatial dimensions(horizontal direction x and vertical direction y), one excitationspectral dimension (λ_(a)), and one emission spectral dimension (λ_(b)).

According to an exemplary embodiment of the present disclosure, ahyperspectral imaging system is described. The system may include asample holder configured to hold a sample, an illumination system, and adetection system.

The illumination system may include a light source configured to emitexcitation light having one or more wavelengths and a first set ofoptical elements. The first set of optical elements may include a firstspatial light modulator (SLM), at least one lens, and at least onedispersive element. The illumination system may be configured tostructure the excitation light into a predetermined two-dimensionalpattern at a conjugate plane of a focal plane in the sample, spectrallydisperse the structured excitation light in a first lateral direction,and illuminate the sample in an excitation pattern with the one or morewavelengths dispersed in the first lateral direction.

The detection system may include a two-dimensional imaging device and asecond set of optical elements. The second set of optical elements mayinclude at least one lens and at least one dispersive element. Thedetection system may be configured to spectrally disperse emission lightcollected from the sample in a second lateral direction and image thespectrally dispersed emission light to the imaging device.

According to a further exemplary embodiment of the present disclosure, amethod for hyperspectral imaging is described. The method includes thesteps of providing a light source that emits excitation light of one ormore wavelengths, structuring, by a first spatial light modulator (SLM),the excitation light from the light source into a predeterminedtwo-dimensional pattern at a conjugate plane of a focal plane in asample, spectrally dispersing, by a first dispersive element, thestructured excitation light in a first lateral direction, illuminatingthe sample in an excitation pattern with the one or more wavelengthsdispersed in the first lateral direction, spectrally dispersing, by asecond dispersive element, emission light collected from the sample in asecond lateral direction, and imaging the spectrally dispersed emissionlight to the imaging device.

According to a yet further exemplary embodiment of the presentdisclosure, a method for configuring a microscope to obtain ahyperspectral-imaging dataset of a sample is described. The methodincludes the steps of providing a light source that emits excitationlight of one or more wavelengths, structuring, by a first spatial lightmodulator (SLM), the excitation light from the light source into apredetermined two-dimensional pattern at a conjugate plane of a focalplane in the sample, spectrally dispersing, by a first dispersiveelement, the structured excitation light in a first lateral direction,illuminating the sample in an excitation pattern with the one or morewavelengths dispersed in the first lateral direction, spectrallydispersing, by a second dispersive element, emission light collectedfrom the sample in a second lateral direction, and imaging thespectrally dispersed emission light to the imaging device.

Additional features and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beobvious from the description, or may be learned by practice of thedisclosed embodiments. The features and advantages of the disclosedembodiments will be realized and attained by the elements andcombinations particularly pointed out in the appended claims.

It is to be understood that both the foregoing general description andthe following detailed description are examples and explanatory only andare not restrictive of the disclosed embodiments as claimed.

The accompanying drawings constitute a part of this specification. Thedrawings illustrate several embodiments of the present disclosure and,together with the description, serve to explain the principles of thedisclosed embodiments as set forth in the accompanying claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a graphical illustration for an exemplary scheme for acquiringa hyperspectral-imaging dataset, according to embodiments of the presentdisclosure.

FIG. 2 is a graphical illustration for another exemplary scheme foracquiring a hyperspectral-imaging dataset, according to embodiments ofthe present disclosure.

FIG. 3 is a schematic representation of an exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 4 is a schematic representation of another exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 5 is a schematic representation of another exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 6 is a schematic representation of another exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 7 is a schematic representation of another exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 8 is a schematic representation of another exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 9 is a schematic representation of another exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 10 is a schematic representation of another exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 11 is a schematic representation of another exemplary hyperspectralimaging system, according to embodiments of the present disclosure.

FIG. 12 is a schematic representation of an exemplary diffractiveelement, according to embodiments of the present disclosure.

FIG. 13 is a schematic representation of another exemplary diffractiveelement, according to embodiments of the present disclosure.

FIG. 14 is a flowchart of an exemplary method for hyperspectral imaging,according to embodiments of the present disclosure.

DETAILED DESCRIPTION

The disclosed embodiments relate to systems and methods for achievinghyperspectral imaging that allows for acquiring a hyperspectral-imagingdataset with both excitation and emission spectra. Embodiments of thepresent disclosure may be implemented using a microscope, such as afluorescence microscope, a confocal microscope (with confocality alongat least one dimension), a transmission microscope, or a reflectancemicroscope, having one or more 2-D imaging devices, e.g., a CCD or CMOSsensor or camera. Alternatively, an optical system may be builtaccording to embodiments of the present disclosure using suitableoptical elements.

Rather than using the time-consuming procedure that acquires ahyperspectral cube for each excitation wavelength, embodiments of thepresent disclosure allow for acquiring a 2-D image of emission spectracorresponding to more than one excitation wavelengths for a subset ofareas on a sample. A plurality of the 2-D images can be acquired andcomputationally reconstructed to obtain a 4-D hyperspectral-imagingdataset of a sample.

According to an aspect of the present disclosure, excitation lighthaving one or more wavelengths may be used to excite fluorophores in thesample. The excitation light may be generated by a multi-color lightsource that emits light with one or more wavelengths. In someembodiments, the multi-color light source may have a continuousspectrum. For example, the multi-color light source may be a broadbandlight source, such as a supercontinuum laser, a white light source(e.g., a high-pressure mercury lamp, a xenon lamp, a halogen lamp, or ametal halide lamp), or one or more LEDs. In other embodiments, themulti-color light source may have a discrete spectrum. For example, themulti-color light source may be a combination of pulsed or continuous“single-wavelength” lasers that emit light with very narrow spectra.

According to an aspect of the present disclosure, excitation lightemitted by the light source may be structured for exciting a subset ofareas on the sample in an excitation pattern using a spatial lightmodulator (SLM). To structure the excitation light, the SLM may modulatethe phase or amplitude of the excitation light by selectively actuatingor switching its pixels. In some embodiments, the SLM may be selectedfrom a group of SLMs including a digital micromirror device (DMD), adiffractive optical element, a liquid crystal device (LCD), and a liquidcrystal-on-silicon (LCOS) device.

According to an aspect of the present disclosure, the structuredexcitation light may be spectrally dispersed in a first lateraldirection (e.g., the vertical direction y or the horizontal directionx). Spectral dispersion of the excitation light may separate or splitone or more wavelengths of the spectrum of the excitation light in thefirst lateral direction. In some embodiments, at least one dispersiveelement may be used to spectrally disperse the excitation light beforeit illuminates the sample in the excitation pattern. The at least onedispersive element may be a diffractive grating or a prism, or acombination of one or more prisms. Therefore, a spectrally dispersedexcitation pattern may be generated to illuminate areas at variousspatial locations on the sample.

Fluorophores or other types of optical labels in the sample may beexcited by the excitation light illuminating the sample. When they relaxto the ground state, the fluorophores or optical labels may emit lightin a range of wavelengths known as the emission spectrum. Thefluorophores or optical labels may have different emission spectracorresponding to different wavelengths of the excitation light.

As described herein, fluorophores are used in this disclosure as anexemplary optical label. Descriptions in references to fluorophores areequally applicable to other types of optical labels consistent with theembodiments of this disclosure. For example, the excitation lightemitted from the light source may also excite other types of opticallabels, which upon excitation, may emit light with an emission spectrum.Therefore, fluorescent light and fluorescence emission spectrum used inthe descriptions in this disclosure may also be used to represent theemission light and emission spectra of other optical labels.

According to an aspect of the present disclosure, fluorescent lightemitted by the fluorophores excited by the excitation light in a givenarea of the sample may be spectrally dispersed in a second lateraldirection (e.g., the horizontal direction x or the vertical directiony). At least one dispersive element may be employed to spectrallydisperse the fluorescent light into a fluorescence emission spectrumcorresponding to the excitation wavelength at that given area. Thefluorescence emission spectra of a subset of areas on the sample may beacquired as a 2-D image in one exposure by the 2-D imaging device.

According to an aspect of the present disclosure, fluorescenceexcitation and emission spectra of all the areas across the sample oracross a field of view may be acquired by scanning the spectrallydispersed excitation pattern in the first and second lateral directionsand acquiring a 2-D image of the fluorescence emission spectra at eachspatial location of the excitation pattern.

In some embodiments, the excitation pattern is scanned across the sampleor the field of view by modulating the pixels of the SLM. In otherembodiments, an x-y translation stage may be used to laterally scan theexcitation pattern across the sample or the field of view by moving thesample or a diffraction grating in the first and second lateraldirections. The stage may be a motorized translation stage, apiezoelectric translation stage, or any suitable stage that allows forlateral linear movement.

Advantageously, the 4-D hyperspectral-imaging dataset may becomputationally reconstructed from the 2-D images of the emissionspectra, each 2-D image corresponding to the excitation pattern at adifferent spatial location on the sample.

In some aspects, systems and methods according to the present disclosureallows for confocal optical sectioning. This may allow for acquisitionof a hyperspectral-imaging dataset for a plurality of focal planes alongan axial direction of the sample. According to an aspect of the presentdisclosure, a hyperspectral-imaging dataset for a focal plane may beacquired by implementing one or more optical pinholes at a planeconjugate to the selected focal plane. The optical pinholes may be oneor more spatial pinholes, or programmable artificial pinholes formed bypixels of a second SLM.

Advantageously, a degree of confocality may be adjusted as needed bychanging the size and/or separation of the artificial pinholes formed bythe SLM. Additionally, a pinhole pattern may be formed by the SLM byselectively modulating or switching its pixels to match the excitationpattern of the excitation light. The pinhole pattern may advantageouslyallow for confocal imaging of a plurality of areas on the samplesimultaneously illuminated by the excitation pattern. This may increasethe speed and/or throughput of acquiring hyperspectral-imaging datasetsacross the sample at the focal plane comparing to traditional confocalmicroscopes that use sequential point-by-point scanning.

Reference will now be made in detail to embodiments and aspects of thepresent disclosure, examples of which are illustrated in theaccompanying drawings. Where possible, the same reference numbers willbe used throughout the drawings to refer to the same or like parts.

As described herein, to illustrate different wavelengths or frequenciesof light, different densities of dotted texture are used in the attacheddrawings. Higher densities correspond to longer wavelengths or lowerfrequencies of light. Additionally, vertical and horizontal directionsare used as examples for illustrating first and second lateraldirections. Alternatively, the horizontal direction may be the firstlateral direction and the vertical direction may be the second lateraldirection. As described herein, any two suitable different directions ora pair of non-parallel, e.g., orthogonal, directions may be used asfirst and second lateral directions.

Exemplary Schemes for Acquiring a Hyperspectral-Imaging Dataset

FIG. 1 graphically illustrates an exemplary scheme for acquiring ahyperspectral-imaging dataset used by methods and systems of the presentdisclosure. As shown in FIG. 1, excitation light having a discretespectrum is illuminated onto a sample in an excitation pattern 100.Excitation pattern 100 may include a 2-D array of excitation spots. Forexample, FIG. 1 illustrates a portion of an exemplary 2-D array ofcircular excitation spots. Additional spots of the array may be locatedabove, below, to the left, and/or to the right of the exemplary array(not shown). As described herein, a suitable size of the array, and asuitable shape, size, and/or separation of the spots may bepredetermined according to the application.

The discrete spectrum of the excitation light includes a plurality ofdiscrete wavelengths or a plurality of narrow spectral bands. Thus, whenthe excitation light is spectrally dispersed by a dispersive elementalong a given lateral direction, excitation pattern 100 may bespectrally dispersed such that different wavelengths of light aredirected to different locations in the given lateral direction. Forexample, as shown in FIG. 1, excitation pattern 100 may include aplurality of scanning cells 110, e.g., a 2-D array of scanning cells110. When the excitation light is spectrally dispersed along thevertical direction, each scanning cell 110 may include a plurality ofexcitation spots 112 a, 112 b, 112 c, 112 d, 112 e, and 112 f verticallyoffset from one another and corresponding to different excitationwavelengths of the excitation light generated by the spectraldispersion.

The vertical separation between the excitation spots may or may not beuniform, and may be predetermined by various factors, such as theexcitation wavelengths, the size of the spots, and the amount ofdispersion of the excitation light. The total number of the verticallydispersed excitation spots in scanning cell 110 may depend on the numberof discrete wavelengths or narrow spectral bands of the excitationlight.

To generate an excitation spectrum of a given spatial location on thesample, spectrally dispersed excitation pattern 100 as shown in FIG. 1may be scanned in the vertical direction such that the excitation spotscorresponding to different excitation wavelengths may be shifted to thisgiven spatial location. For example, when excitation pattern 100 isshifted in the vertical direction, areas in each scanning cell 110previously illuminated by excitation spots 112 a, 112 b, 112 c, 112 d,112 e, and 112 f can be illuminated by different ones of theseexcitation spots. For instance, by shifting excitation pattern 100 overone excitation spot, the areas previously illuminated by excitationspots 112 b are illuminated by excitation spots 112 a, the areaspreviously illuminated by excitation spots 112 c are illuminated byexcitation spots 112 b, the areas previously illuminated by excitationspots 112 d are illuminated by excitation spots 112 c, the areaspreviously illuminated by excitation spots 112 e are illuminated byexcitation spots 112 d, the areas previously illuminated by excitationspots 112 f are illuminated by excitation spots 112 e, and the areaspreviously illuminated by excitation spots 112 a are illuminated byexcitation spots 112 f shifted from scanning cells located above (notshown).

Areas within each scanning cell 110 may be scanned by shiftingspectrally dispersed excitation pattern 100 in the vertical andhorizontal directions. For example, by shifting excitation pattern 100over the length of scanning cell 110 in the vertical direction, a givenarea in scanning cell 110 can be illuminated by the different excitationspots corresponding to the different excitation wavelengths of the lightsource. By shifting excitation pattern 100 vertically and/orhorizontally in a continuous fashion or at predetermined separations(e.g., based on the desired vertical and/or horizontal resolution) overthe lengths of scanning cell 110, each given area in scanning cell 110can be illuminated by the different excitation spots.

As shown in FIG. 1, the excitation spots of excitation pattern 100 areseparated in the horizontal direction at a given period. Advantageously,the periodic separation of the excitation spots in the horizontaldirection allows for measuring the fluorescence emission spectra of theexcited areas on the sample. For example, the emitted fluorescent lightfrom a given area illuminated by an excitation spot can be spectrallydispersed in the horizontal direction without overlapping with that ofanother area. Therefore, the fluorescence emission spectra of aplurality of areas simultaneously illuminated by the excitation spotscan be generated and acquired by a 2-D imaging sensor or device.

FIG. 1 shows a 2-D image 200 of the fluorescence emission spectra withthe excitation wavelengths (λ_(a)) represented in the vertical directionand the emission wavelengths (λ_(b)) represented in the horizontaldirection. FIG. 1 graphically illustrates that, in 2-D image 200, areasexcited by excitation spots 112 a, 112 b, 112 c, 112 d, 112 e, and 112 fcorresponding to different excitation wavelengths may generate differentfluorescence emission spectra 212 a, 212 b, 212 c, 212 d, 212 e, and 212f extending in the horizontal direction and correspondingly offset fromone another in the vertical direction. Therefore, a plurality offluorescence emission spectra can be acquired in 2-D image 200, witheach emission spectrum corresponding to an excited spot of excitationpattern 100 at a different spatial location on the sample.

As described above, different areas in each scanning cell 110 may beilluminated by spatially shifting excitation pattern 100 laterally inthe vertical and horizontal directions. At each spatial position ofexcitation pattern 100, fluorescence emission spectra of the illuminatedareas can be acquired on 2-D image 200. Therefore, a plurality of 2-Dimages 200 of fluorescence emission spectra may be acquiredcorresponding to a series of excitation patterns 100 laterally shiftedfrom one another.

By combining datasets of the acquired 2-D images 200, a fluorescenceexcitation-emission matrix (EEM) may be acquired for each pixel orspatial location in the 2-D images 200. The fluorescence EEM may recordor display fluorescence intensities as a function of a plurality ofexcitation wavelengths and a range of emission wavelengths. Therefore, a4-D hyperspectral-imaging dataset of the sample having both theexcitation and emission spectra may be collected and reconstructed fromthe acquired 2-D images 200.

FIG. 2 graphically illustrates another exemplary scheme for acquiring ahyperspectral-imaging dataset by methods and systems of the presentdisclosure. As shown in FIG. 2, when the excitation light has acontinuous spectrum and is spectrally dispersed by a dispersive elementalong the vertical direction, a focused spot of the excitation light maybe spectrally dispersed into a continuous excitation line along thevertical direction. In such instances, as shown in FIG. 2, excitationpattern 100 may be spectrally dispersed into a 2-D array of excitationlines 114, each representing the range of wavelengths of the excitationlight vertically offset in a continuous fashion.

FIG. 2 illustrates a portion of an exemplary 2-D array of excitationlines 114, which shows a three-by-three 2-D array. Other excitationlines of the array may be located above, below, to the left, and/or tothe right of the exemplary array (not shown). As described herein, asuitable size of the array, a suitable shape, size, and/or separation ofthe excitation lines may be selected according to a given application.

Areas within each scanning cell 110 may be similarly scanned asdescribed above by shifting spectrally dispersed excitation pattern 100in the vertical and horizontal directions. An array of fluorescenceemission spectra 214 corresponding to the array of excitation lines 114of excitation pattern 100 may be similarly acquired on 2-D image 200.Each fluorescence emission spectrum 214 in 2-D image 200 corresponds toa continuous strip on the sample illuminated by an excitation line 114of excitation pattern 100.

In the scheme shown in FIG. 2, in some embodiments, when shiftingspectrally dispersed excitation pattern 100 in the vertical direction,an area excited at a first wavelength along excitation line 114 may thenbe excited at a second wavelength longer or shorter than the firstwavelength after the vertical shifting. Therefore, by shiftingexcitation pattern 100 vertically in a continuous fashion or overpredetermined separations (e.g., based on the desired verticalresolution), areas illuminated by excitation lines 114 in each scanningcell 110 can be illuminated in the different excitation wavelengths ofthe light source. Similar to the scheme shown in FIG. 1, other areas onthe sample in each scanning cell 110 may be illuminated by the differentexcitation lines by shifting excitation pattern 100 in the horizontaldirection.

As described herein, the areas on the sample illuminated by excitationpattern 100 may be substantially determined by the size and shape of theexcitation spots or excitation lines of excitation pattern 100. The sizeand shape of the excitation spots or excitation lines may be determinedby many factors of the optical system, including the size and shapes ofthe pixels of the SLM, the magnification of the optical system, and thedegree of spectral dispersion of the excitation light.

The spatial separation, horizontal and/or vertical, between excitationspots or lines of excitation pattern 100 may be predetermined based onvarious factors, such as the excitation wavelengths, the size of thesample, the field of view of the optical system, the desired measurementthroughput, spatial resolution, and/or speed, and the amounts ofspectral dispersion of excitation light and/or emitted fluorescentlight.

For example, the spatial separation between the excitation spots orlines in the vertical direction may be predetermined based on the amountof spectral dispersion of the excitation light such that the excitationspots or lines do not overlap in the vertical direction. The spatialseparation between the excitation spots or lines in the horizontaldirection may be predetermined based on the range of the fluorescenceemission spectra in the horizontal direction such that the fluorescenceemission spectra do not overlap with each other in the horizontaldirection.

In some embodiments, the horizontal and/or vertical periods of an arrayof excitation spots for different wavelengths may be the same. In otherembodiments, the horizontal and/or vertical periods of an array ofexcitation spots for different wavelengths may be different. Differentspatial periods may be convenient for computationally reconstructing the4-D hyperspectral imaging dataset in some cases, for example, where theSLM is placed at a Fourier plane of the sample to generate excitationpattern 100 as described further below.

Embodiments to be described below in reference to schematicrepresentations of optical systems and/or components are directed tosystems and methods for achieving the above-described schemes foracquiring a 4-D hyperspectral-imaging dataset. The schematicrepresentations are to be understood as not being drawn to scale.

Exemplary Optical Systems and Components

FIG. 3 is a schematic representation of an exemplary hyperspectralimaging system 300. In some embodiments, system 300 may be afluorescence microscope, a transmission microscope, or a reflectancemicroscope, or a confocal fluorescence microscope(with confocality alongat least one dimension). Embodiments of the present disclosure areapplicable to other suitable microscopy techniques for performinghyperspectral imaging.

As shown in FIG. 3, system 300 may include an illumination system and adetection system, each having a plurality of components. Theillumination system may include a light source 310, a first SLM 320 a,at least one lens, e.g., lens 330 a, and at least one dispersiveelement, e.g., dispersive element 340 a. The detection system mayinclude a 2-D imaging device 380, at least one lens, e.g., lens 330 b,and at least one dispersive element, e.g., dispersive element 340 b.Depending on its layout, geometry, and/or application, system 300 mayfurther include a beamsplitter 350, an objective 360, a polarizer 390 a,and/or a sample holder 370 where a sample to be imaged is placed. System300 may further include other optical elements, such as mirrors, beamdumps, etc.

As described herein, an optical axis of system 300 may define a pathalong which the excitation light and emitted fluorescent light from thesample propagate through system 300.

In the illumination system, as shown in FIG. 3, light source 310 emitsexcitation light 402, which is directed to SLM 320 a. Excitation light402 may be collimated and/or expanded using one or two lenses (notshown). SLM 320 a may structure excitation light 402 through modulatingthe phase or amplitude of excitation light 402 by selectively actuatingor switching its pixels. At least a portion of the pixels of SLM 320 areflect excitation light 402 and direct it along the optical axis ofsystem 300.

As shown in FIG. 3, reflected excitation light 404 transmits throughlens 330 a and dispersive element 340 a. Lens 330 a may collimatereflected excitation light 404 along the optical axis. Dispersiveelement 340 a spectrally disperses reflected excitation light 404 alonga first lateral direction. For example, dispersive element 340 a maycause small wavelength-dependent angular deflection to reflectedexcitation light 404. Spectrally dispersed excitation light 406 may bereflected by beamsplitter 350 and directed towards objective 360.Objective 360 then focuses spectrally dispersed excitation light 406 toa sample placed on sample holder 370.

In the detection system, as shown in FIG. 3, fluorescent light emittedby excited fluorophores in the sample is collected and/or collimated byobjective 360. Fluorescent light 408 transmits through beamsplitter 350and dispersive element 340 b along the optical axis of system 300.Dispersive element 340 b spectrally disperses fluorescent light 408along a second lateral direction as described above. Spectrallydispersed fluorescent light 410 transmits through lens 330 b and isacquired by 2-D imaging device 380. Imaging device 380 may be placedabout one focal length away from lens 330 b such that lens 330 b mayimage and focus the spectrally dispersed fluorescent light 410 onto a2-D sensor of imaging device 380.

Other configurations of system 300 are possible using additional opticalelements, such as mirrors, lenses, etc., as further described below.

Functions and the working principles of various components of system 300are described in detail below.

Light Source

As described above, light source 310 may have a continuous spectrum or adiscrete spectrum. Light source 310 may be a white light source, such asa supercontinuum laser, or a combination of “single-wavelength” laserswith discrete narrow spectral bands. In some embodiments, excitationlight 402 emitted by light source 310 may be directed straight towardsSLM 320 a. In other embodiments, excitation light 402 may be collimatedand/or expanded by lenses before being incident on SLM 320 a.Additionally or alternatively, excitation light 402 may be diffusedusing a diffuser or a despeckling element to reduce the speckle effectof coherent illumination.

In some embodiments, light source 310 may be operably connected to acontroller (not shown) having a processor and a computer-readable mediumthat stores instructions or operational steps. These instructions orsteps, when executed by the processor, modulate the operational statesof light source 310. For example, the processor may activate ordeactivate light source 310, modulate the duration of a pulse when lightsource 310 is a pulsed light source, and/or switch or tune the emissionwavelengths of light source 310.

Spatial Light Modulator for Modulating Excitation Light

As described above, to structure excitation light 402 for illuminatingthe sample in excitation pattern 100, SLM 320 a may modulate theamplitude or phase of excitation light 402 by selectively modulating itspixels between operational states.

Amplitude Modulation

In some embodiments, the amplitude of excitation light 402 may bemodulated by SLM 320 a. For example, SLM 320 a may be a digitalmicromirror device (DMD) having an array of multiple micromirrors (notshown). These mirrors may be individually actuated to switch between twooperational positions, an “on” position and an “off” position. When amicromirror is configured to be in the “on” position, excitation light402 is reflected to propagate along the optical axis as reflectedexcitation light 404 directed to the sample. When a micromirror isconfigured to be in the “off” position, excitation light 402 isreflected towards a direction deviated from the optical axis and is notdirected to the sample (not shown). In some embodiments, excitationlight 402 reflected by the “off” micromirrors may be directed to otheroptical elements, such as a mirror or a beam dump (not shown).

In some embodiments, the micromirrors are of a square shape having alength of its sides ranging from about a few micrometers to about 10 μm.Other shapes and sizes of the micromirrors are also possible and may besuitably used. The DMD is typically capable of changing or alternatingthe “on” and “off” positions of the micromirrors very rapidly.

In some embodiments, a single micromirror of the DMD may be referred toas a single pixel. In other embodiments, a plurality of micromirrors maybe referred to as a single pixel. For example, a group of immediatelyadjacent micromirrors may be referred as a single pixel and may bemodulated or actuated to the same position.

An amplitude modulation pattern may be formed by the micromirrors orpixels of the DMD in the “on” position. The amplitude modulation patternmay be imaged onto the sample as excitation pattern 100 by lens 330 aand objective 360. For example, lens 330 a is used as a tube lens andcombined with objective 360 to form an imaging configuration. The DMD isplaced at a conjugate plane to the sample or at about one focal lengthbefore lens 330 a. Depending on the focal lengths of lens 330 a andobjective 360, excitation pattern 100 may be a magnified or de-magnifiedimage of the amplitude modulation pattern.

In other embodiments, to modulate the amplitude of excitation light 402,SLM 320 a may be a liquid crystal device (LCD) or a liquidcrystal-on-silicon (LCOS) device. Pixels of SLM 320 a may create anamplitude modulation pattern by manipulating the polarization of lightincident on the pixels. Similar to the DMD, the LCD or LCOS device maybe placed at a conjugate plane to the sample. Pixels of the LCD or LCOSdevice may be electrically modulated between an “on” state and an “off”state in a pixel-by-pixel fashion. The “on” pixels may rotate theorientation of linearly polarized light by about 90° while the “off”pixels do not perform the rotation. In such instances, a first linearpolarizer (not shown) may be used to linearly polarize excitation light402. A second linear polarizer or a polarizing beamsplitter (PBS) (notshown) may be used to transmit excitation light 404 reflected by the“on” pixels and block excitation light 402 reflected by the “off”pixels.

A disadvantage of modulating the amplitude of excitation light 402 usingSLM 320 a is the loss of light during the modulation. This is becausemost of the pixels of SLM 320 a are typically in the “off” state.Accordingly, most of excitation light 402 is steered away from theoptical axis and would not reach the sample, and thus is lost.Excitation light recycling systems may be used to reduce this loss byredirecting off-optical axis excitation light back to the optical axisas described further below.

Phase Modulation

In some embodiments, the phase of excitation light 402 may be modulatedby SLM 320 a. SLM 320 a may be a reflection type LCD or LCOS device.FIG. 4 is a schematic representation of an exemplary system 300 using aLCD or LCOS device as SLM 320 a. As shown in FIG. 4, the LCD or LCOSdevice may be placed close to a conjugate plane 322 to the sample. Acustom phase modulation pattern may be formed by the pixels of the LCDor LCOS device. The phase modulation pattern may create an array ofoff-axis lens phase profiles. Wavefront of excitation light 402 may thenbe modulated by the phase modulation pattern and form a preliminaryexcitation pattern (e.g., a diffraction pattern) at conjugate plane 322.This preliminary excitation pattern may be a magnified or de-magnifiedimage of excitation pattern 100 and may include an array of focusedspots.

Conjugate plane 322 may be located a short distance beyond SLM 320 a.The focal plane of the focused spots of the preliminary excitationpattern may be wavelength dependent. Therefore, different wavelengths ofexcitation light 402 may not all focus on conjugate plane 322. In someembodiments, the focal plane for the center wavelength of excitationlight 402 is approximately at conjugate plane 322. The preliminaryexcitation pattern formed at or close to conjugate plane 322 is thenimaged onto the sample as excitation pattern 100 by lens 330 a andobjective 360. Although different wavelengths of excitation pattern 100in this configuration may have slightly different focal planes,modulating the phase of excitation light 402 increases the efficiency ofusing excitation light 402 comparing to amplitude modulation.

In other embodiments, the LCD or LCOS device may be placed at anaperture plane, which may be a conjugate plane to the back aperture ofobjective 360 or a Fourier plane to the sample. For example, oneexemplary configuration of system 300 may have two tube lenses (notshown) placed between SLM 320 a and objective 360. A first tube lens maybe located about one focal length behind SLM 320 a. A second tube lensmay be located about two focal lengths behind the first tube lens.Objective 360 may be located about one focal length behind the secondtube lens.

The pixels of the LCD or LCOS device may form a custom phase modulationpattern to modulate the wavefront of excitation light 402. Upon thereflection of excitation light 402 by the LCD or LCOS device, phases atdifferent locations of the wavefront of the reflected excitation light404 may be selectively changed according to the phase modulationpattern. In some embodiments, pixels of the LCD or LCOS device may beelectrically modulated between an “on” state and an “off” state in apixel-by-pixel fashion. If pixels of the LCD or LCOS device are in the“on” state, they may change the phase of the reflected light by changingthe optical path length of light traveled in the liquid crystal; and ifthey are in the “off” state, they may not change the phase of thereflected light. This allows the phase modulation pattern formed by thepixels to be digitally customized as needed. In other embodiments,pixels of the LCD or LCOS device may have multiple states or levels ofadjustment (e.g., 256 levels) and may be individually modulated todesired states or levels. Advantageously, increasing the states orlevels of adjustment of the pixels increases the continuity of theadjustment of the phase modulation pattern and thus the adjustment ofthe phase of excitation light 402.

The phase modulation may render wavelets of reflected excitation light404 having different directions and/or phases. As reflected excitationlight 404 propagates along the optical axis, each of the tube lenses andobjective 360 may perform Fourier Transform on the wavefront ofreflected excitation light 404. A diffraction pattern may then be formedat the focal plane of objective 360. This diffraction pattern isreferred to herein as excitation pattern 100 when illuminated on thesample.

In the above-described configuration, because the phase modulationpattern is at or approximately at a Fourier plane to the sample, thephase modulation pattern is the inverse Fourier transform of a desiredexcitation pattern 100 illuminated on the sample. Because FourierTransform includes a scaling factor proportional to the wavelength oflight, the spatial periods of the array of excitation spots fordifferent wavelengths in excitation pattern 100 may be different. Forexample, longer wavelength would diffract at larger angles, which can beconverted to larger spatial periods. This may cause the correspondingfluorescence emission spectra arrays acquired in 2-D image 200 to havedifferent spatial periods. Customized computer algorithms may be usedfor generating time-varying phase modulation patterns for scanningacross the field of view and/or for computationally reconstructing the4-D hyperspectral-imaging dataset from datasets of such 2-D images.

Advantageously, modulating the phase of excitation light 402 may allowit to propagate with substantially uniform intensity in the near fieldof the LCD or LCOS device and thus reduce loss of light. The modulatedexcitation light may then form customizable or programmable excitationpattern 100 on the sample in the far field. Therefore, comparing tomodulating the amplitude of excitation light 402 as described above,modulating the phase of excitation light 402 to create excitationpattern 100 may substantially increase the efficiency of illumination ofsystem 300 by reducing loss of excitation light 402. Additionally,increasing the continuity of the phase modulation of excitation light402 may further increase the diffraction efficiency of the LCD or LCOSdevice and thus the efficiency of illumination of system 300.

The LCD or LCOS device for modulating the amplitude or phase ofexcitation light 402 may alternatively be a transmission type deviceimplemented along the optical axis. The geometry of the illuminationsystem may be suitably designed such that the amplitude or phasemodulation pattern formed by the pixels of the device may modulate theamplitude or phase of excitation light 402 similarly as described above.

Whether SLM 320 a modulates the amplitude or phase of excitation light402, excitation pattern 100 can be programmed and customized as neededby modulating pixels of SLM 320 a between two or multiple operationalstates or levels in a pixel-by-pixel fashion. Further, excitationpattern 100 can be translated or shifted in a given spatial direction,such as the horizontal or vertical direction, by scanning or shiftingthe modulation of the pixels of SLM 320 a. This advantageously allowsfor scanning excitation pattern 100 across the field of view of system300 without moving the sample and/or sample holder 370 using an x-ytranslation stage.

In some embodiments, depending on the type and modulation features ofthe pixels of SLM 320 a, excitation light 402 may be directed towardsSLM 320 a at a predetermined angle relative to a plane of SLM 320 a. Thepredetermined angle may depend on the type of SLM 320 a and/or thegeometry of system 300. In some instances, when SLM 320 a is areflection type SLM that modulates the phase of excitation light 402,excitation light 402 may be directed towards SLM 320 a at an angle suchthat reflected excitation light 404 propagates along the optical axis ofsystem 300. In other instances, when SLM 320 a is a DMD, excitationlight 402 may be directed towards the DMD at an angle such thatexcitation light 404 reflected by the “on” micromirrors propagates alongthe optical axis.

In some embodiments, SLM 320 a may be operably connected to a controller(not shown) having a processor and a computer-readable medium thatstores instructions or operational steps. These instructions or steps,when executed by the processor, modulate the operational states of thepixels of SLM 320 a to form a desired excitation pattern 100 and/or totranslate excitation pattern 100 in a desired spatial direction over apredetermined distance across the field of view.

Lenses and Objective

Various lenses of system 300, such as lenses 330 a and 330 b, may beachromatic, such as achromatic doublets or triplets, to limit or reducethe effects of chromatic and/or spherical aberration of the system.Further, objective 360 of system 300 may be achromatic. Alternatively oradditionally, objective 360 may be an infinity-corrected objective suchthat objective 360 may form a desired focus (e.g., focused spots orfocused pattern) of a collimated light beam entering from its backaperture. Using achromatic lenses and/or achromatic orinfinity-corrected objective may allow excitation light 402 of differentwavelengths to have at least approximately the same focus in the sample.Further, using achromatic lenses and/or achromatic objective may allowfluorescent light of different wavelengths from a focal plane in thesample to similarly form a focused image at imaging device 380.Therefore, using achromatic lenses and/or achromatic objective mayimprove the quality of 2-D images 200 of fluorescence emission spectra,and thus the quality of the reconstructed hyperspectral-imaging dataset.

Dispersive Elements

Dispersive elements 340 a and 340 b may be diffraction gratings orprisms, such as non-deviating prisms (e.g., Amici prisms or double Amiciprisms). The types of dispersive elements 340 a and 340 b may be thesame or may be different. The degree of dispersion caused by dispersiveelements 340 a and 340 b may be same or different, and may bepredetermined based on various factors, such as the spectral ranges ofexcitation light and fluorescent light, the size of the sample or thefield of view, the size of imaging device 380, the desired spectralresolution, and the application of system 300.

In some embodiments, the degree of dispersion caused by dispersiveelements 340 a and 340 b may be adjustable. For example, dispersiveelement 340 a may be a pair of double Amici prisms placed along theoptical axis of system 300. At least one of the pair of double Amiciprisms is rotatable relative to the other around the optical axis. Therotation of the double Amici prisms relative to each other may allow forcontinuous control of the amount and/or angular orientation of thespectral dispersion of excitation light 402. Similarly, dispersiveelement 340 b may be a pair of double Amici prisms, allowing forcontinuous control of the amount and/or angular orientations of thespectral dispersion (e.g., dispersion angles) of fluorescent light 408.

Excitation Light Blocking

Because the intensity of excitation light 402 may be orders of magnitudestronger than fluorescent light 408, excitation light 402 reflectedand/or scattered by the sample and/or sample holder 370 may enter thedetection system and affect the detection or acquisition of thefluorescence emission spectra by imaging device 380. Therefore,embodiments of the present disclosure may reduce or block excitationlight 402 from propagating into the detection system as described below.

In some embodiments, beamsplitter 350 may be used to reject or blockexcitation light 402 from propagating into the detection system. Forexample, beamsplitter 350 of system 300 may be a dichroic beamsplitter,a polarizing beamsplitter (PBS), or other suitable type of beamsplitter.

When light source 310 or excitation light 402 has a discrete spectrumhaving one or more discrete wavelengths or narrow spectral bands,beamsplitter 350 may be a dichroic beamsplitter that selectivelyreflects and transmits light depending on its wavelength. For example,beamsplitter 350 may be a multiband dichroic that has multiple cut-offwavelengths and passbands. The multiband dichroic may be selected tosubstantially reflect wavelengths of excitation light 402 and tosubstantially transmit wavelengths of fluorescent light 408. In suchinstances, some wavelengths of fluorescent light 408 that are the sameor close to that of excitation light 402 may be substantially blocked,and thus may have substantially reduced intensity in 2-D image 200acquired by imaging device 380.

Alternatively or additionally, when light source 310 or excitation light402 has a discrete spectrum, a set of corresponding notch filters or asingle multi-notch filter (not shown) may be added to the detectionsystem along the optical axis. The notch filters or multi-north filtermay selectively reflect the discrete wavelengths or narrow spectralbands of excitation light 402, thereby blocking excitation light 402from reaching imaging device 380. Again, some wavelengths of fluorescentlight 408 that are the same or close to that of excitation light 402 maybe substantially blocked by the notch filters, and thus may havesubstantially reduced intensity in 2-D image 200 acquired by imagingdevice 380.

When light source 310 or excitation light 402 has a continuous spectrum,beamsplitter 350 may be a long-pass dichroic beamsplitter that reflectsat least a portion of the wavelengths of excitation light 402 andtransmits at least a portion of the wavelengths of fluorescent light408. The spectrum of excitation light 402 typically ranges from theultraviolet through the visible spectra, and the spectrum of fluorescentlight 408 typically ranges from the visible into the near infraredspectra. Therefore, the long-pass dichroic beamsplitter may blockwavelengths of excitation light 402 and transmit wavelengths offluorescent light 408. However, in some instances, both the spectrum ofexcitation light 402 and spectrum of fluorescent light 408 may includeshort to long wavelengths and they may overlap, e.g., in the visiblespectrum. In such instances, the long-pass dichroic beamsplitter mayblock at least some fluorescent light 408 in the visible spectrum, andmay not be suitable for rejecting excitation light 402 in applicationswhere the blocked spectrum of fluorescence light 408 contains desiredspectral information, for example.

Regardless of the types of spectrum of light source 310 or excitationlight 402 (whether or not discrete or continuous), in some embodiments,polarizer 390 a and beamsplitter 350 may be used in combination to blockexcitation light 402 from entering the detection system and thus frompropagating towards imaging device 380. For example, beamsplitter 350may be a polarizing beamsplitter (PBS) that reflects light whosevibration orientation aligns with the transmission axis of polarizer 390a.

For example, polarizer 390 a may be placed at any suitable locationalong the optical axis to linearly polarize excitation light 402. ThePBS may be selected to reflect light having a vibration orientation sameas that of the polarized excitation light and to transmit light having avibration orientation perpendicular to that of the polarized excitationlight. Most of the excitation light collected by objective 360 wouldtherefore reflect from this PBS and would not reach imaging device 380.In some instances, both the sample and objective 360 may depolarizereflected and/or scattered excitation light to a small degree, and thusundesirably allow some excitation light to transmit through the PBS andenter the detection system.

2-D Imaging Device

Imaging device 380 may include a suitable 2-D sensor located at an imageplane conjugate to a selected focal plane in the sample. The sensorcould be implemented with a CMOS sensor, a CCD sensor, a 2-D array ofsilicon avalanche photodiodes (APDs), an electron-multiplied CCD(EMCCD), an intensified CCD, or other suitable types of 2-D sensors.

Imaging device 380 may be operatively connected to a controller or acomputing device (not shown) that controls its operation. For example,controller (not shown) may have a processor and one or morecomputer-readable medium that stores instructions or operational steps.The instructions or operational steps, when executed by the processor,may operate the exposure of imaging device 380, acquire 2-D images 200,and/or store the datasets of 2-D image 200 to a memory. Thecomputer-readable medium may further store instructions or operationalsteps that, when executed by the processor, perform data processing ofthe acquired 2-D image datasets and/or reconstruct the 4-Dhyperspectral-imaging dataset from the 2-D image datasets.

System 300 may advantageously have additional technical features andcapabilities to enhance its functionality and performance as describedin detail below.

Time-Resolved Capability

In some embodiments, time-resolved capability may be advantageouslyadded to system 300 to allow for fluorescence lifetime imaging (FLIM) ortime-resolved fluorescence spectroscopy. For example, a pulsed lightsource, such as a supercontinuum laser, may be used as light source 310,together with a 2-D imaging device 380 having picosecond to nanosecondtime-gating capability, such as an intensified CCD camera or anoptoelectronic streak camera. Alternatively, a conventional 2-D CCD orCMOS sensor may be used in combination with an electro-optic shutter. Insome embodiments, a modulated, electron-multiplied fluorescence lifetimeimaging microscope (MEM-FLIM) camera may be used in combination with amodulated light source 310, e.g., a pulsed light source.

The lifetime of the fluorophores or fluorescent molecules in the samplemay be calculated from the acquired time-resolved 2-D images of thefluorescence emission spectra for each spatial location in the field ofview. This adds another dimension of information to thehyperspectral-imaging dataset, thereby providing additional informationabout the fluorophores or fluorescent molecules in the sample.

Because FLIM excites the fluorophores with short excitation pulses inthe time-domain, the FLIM capability of system 300 may substantiallyreject the scattered and/or reflected excitation light by discarding thesignals close to zero delay. This may advantageously reduce or minimizethe effect of the scattered and/or reflected excitation light in theacquired fluorescence signals, e.g., 2-D image 200.

Fluorescence Polarization

In some embodiments, system 300 may advantageously allow forfluorescence polarization (or anisotropy) measurement to obtainadditional information about the fluorophores or fluorescent moleculesin the sample. Relationships between the polarization of the excitationlight and the emitted fluorescent light subsequently detected may beused to analyze and study various chemical and/or physical processes ofthe molecules in the sample, such as rotational diffusion, bindinginteractions, and orientation.

To add the capability for measuring fluorescence polarization, as shownin FIG. 5, system 300 may include polarizer 390 a and an opticalelement, such as a waveplate or a polarizer. For example, the opticalelement may be an achromatic half-wave plate (HWP) 390 b. Polarizer 390a may be a linear polarizer located in the illumination system, therebygenerating linearly polarized excitation light, e.g., verticallypolarized light. Depending upon the orientation of their absorptiondipoles, individual fluorophores in the sample are preferentiallyexcited by the linearly polarized excitation light. The fluorescentlight emitted from the sample may be partially depolarized due to randomorientation, diffusion, and/or rotation of the fluorophores.

Beamsplitter 350 may be a polarizing beamsplitter (PBS) thatsubstantially transmits horizontally polarized light and reflectsvertically polarized light. For example, as shown in FIG. 5, theexcitation light vertically polarized by polarizer 390 a can bereflected by the PBS and then propagates towards HWP 390 b. HWP 390 bmay be placed at a suitable location, such as before beamsplitter 350along the optical axis. In such instances, HWP 390 b may rotate thevibration orientations of both the linearly polarized excitation lightand the collected fluorescent light from the sample by about twice theangle between a vertical axis and the fast axis of the HWP, for example.Rotating HWP 390 b around the optical axis would advantageously rotatethe vibration directions of both the excitation light and fluorescentlight. Beamsplitter 350 may substantially block the polarized excitationlight and transmit at least a portion of polarized fluorescent light tobe acquired by imaging device 380.

Depending on the application, such fluorescence polarization assays maybe performed in steady state or with time-resolved measurements, such asutilizing the FLIM capability as described above.

Measurement of fluorescence polarization (or anisotropy) adds anotherdimension of information to the hyperspectral-imaging dataset acquiredby system 300. This additional dimension of information may complementthe information in the other dimensions of the hyperspectral-imagingdataset about the local chemical and physical environments offluorophore-tagged molecules in the sample, such as molecular mass andorientation of the molecules. The augmented hyperspectral-imagingdataset acquired by system 300 may further improve the accuracy ofdiagnosis of physiologic or pathologic changes of the sample.

Excitation Light Recycling System

As described above, because most of excitation light 402 is steered awayfrom the optical axis and would not reach the sample, modulating theamplitude of excitation light 402 using SLM 320 a, e.g., a DMD or a LCD,to generate excitation pattern 100 results in loss of light. Therefore,in some embodiments, system 300 may advantageously include an excitationlight recycling system 500 to increase efficiency of utilization ofexcitation light 402. Recycling system 500 may redirect the off-opticalaxis excitation light back to the optical axis towards the sample asdescribed below.

Reflection-Based Scheme

In some embodiments, excitation light recycling system 500 uses areflection-based scheme as shown in FIG. 6. Recycling system 500 mayinclude one or more lenses and mirrors. For example, recycling system500 may include a lens 330 c, a flat mirror 510, a first concave mirror520 a, and a second concave mirror 520 b. The concave mirrors may bereplaced by a combination of a flat mirror and a lens.

Excitation light 402 may be collimated before passing lens 330 c. Lens330 c may focus collimated excitation light 402 to a focal point 312 aof mirror 520 a at a given plane 312 near or aligned with the plane ofSLM 320 a. Then, excitation light 402 expands as it propagates fromfocal point 312 a to mirror 520 a. Mirror 520 a may re-collimateexcitation light 402 and direct it to SLM 320 a.

As described above, when SLM 320 a is a DMD, a small fraction ofexcitation light 402 may be reflected by the “on” pixels towards lens330 a along the optical axis, while the rest, e.g., off-axis excitationlight 403, is reflected by the “off” pixels at a different angle andaway from the optical axis. Mirror 520 b may be configured to interceptthis off-axis excitation light 403 and reflect it back to a point 312 bvery close to focal point 312 a. The separation between point 312 b andfocal point 312 a may be just large enough to allow the edge of mirror510 to intercept the returned off-axis excitation light 403 withoutsubstantially blocking the original excitation light 402. Mirror 510then may direct off-axis excitation light 403 back to SLM 320 a via apath that is nearly aligned with the original path. In suchconfiguration, excitation light 402 can be incident onto SLM 320 a manytimes through multiple reflections between the mirrors, therebyrecycling off-axis excitation light 403 back to the optical axis.

As described herein, the three paths for the recycling of off-axisexcitation light 403 shown in FIG. 6 are exemplary only. Multiple orinfinite recycling paths may be possible.

A few design considerations of system 300 are discussed in thefollowing. In some instances, the recycled off-axis excitation light 403may be slightly divergent. For each recycling path of off-axisexcitation light 403 propagating in recycling system 500, becauseoff-axis excitation light 403 is not returned to focal point 312 a,off-axis excitation light 403 would have a slightly different angle whenit reaches SLM 320 a from that of the original excitation light 402. Theangular difference (or divergent angle) may be defined as Δθ=Δx/f, where“Δx” is the separation between focal point 312 a and point 312 b, and“f” is the focal length of mirror 520 a (or a lens) for re-collimatingthe off-axis excitation light reflected by mirror 510. Δx may be atleast greater than any unusable rough edge of mirror 510, and greaterthan the diffraction limited spot size of excitation light 402.Depending on the values of Δx and f, Δθ may be less than 1 degree. Suchsmall degree of angular difference (or divergence angle) may not affectthe formation of excitation pattern 100.

In some instances, when SLM 320 a is a DMD, the DMD may have adiffraction effect on reflected excitation light 404. For example, asingle micromirror of the DMD may have a side length of approximately 10μm. A typical divergence angle for reflected excitation light 404 causedby the diffraction of the micromirror array may be about λ_(a)/10 μm,where λ_(a) is the wavelength of excitation light 402. Therefore, thedivergence angle may be about less than one radian, e.g., 1/20 radian,or less than a few degrees, e.g., 3 degrees. Thus, most of excitationlight 404 reflected by the “on” pixels or micromirrors of the DMD fromdifferent recycling paths in recycling system 500 may overlap andpropagate along the optical axis, and thus may not affect the formationof excitation pattern 100.

In some instances, reflected excitation light 404 from differentrecycling paths in recycling system 500 may exhibit opticalinterference. For a light source 310 having discrete wavelengths ornarrow spectral bands, this interference may cause reflected excitationlight 404 to have unstable intensities when focused on the sample.Additional optical components may be added to control the relativephases of excitation light 403 propagating in different recycling pathsto reduce the optical interference effect. However, this may complicatethe design of system 300. Therefore, the reflection-based scheme shownin FIG. 6 for recycling system 500 may be more suitable for systems 300having a light source 310 with a broadband spectrum, such as a whitelight source. For such systems 300, the effect of the interference mayimpose very rapid small oscillations on the spectrum of reflectedexcitation light 404. Fluorophores typically have spectrally broadabsorption features, allowing these oscillations to average out duringexcitation. Therefore, the optical interference effect may have littleeffect on the acquired fluorescence emission spectra when light source310 has a broadband spectrum.

Polarization-Based Scheme

To solve the above-described technical problem for recycling excitationlight 402 having discrete wavelengths or narrow spectra bands, in someembodiments, excitation light recycling system 500 may use apolarization-based scheme as shown in FIG. 7.

As shown in FIG. 7, polarization-based recycling system 500 may includeone or more optical components. For example, recycling system 500 mayinclude an optical isolator 530, a polarizing beamsplitter (PBS) 540, aquarter-wave plate (QWP) 550, and one or more mirrors, e.g., a firstmirror 510 a and a second mirror 510 b. In some embodiments, opticalisolator 530 may include a linear polarizer or may be optionallyreplaced by a linear polarizer.

In this scheme, optical isolator 530 allows the propagation ofexcitation light 402 in only one forward direction. Excitation light 402may be a linearly polarized light, or may become linearly polarizedafter passing through optical isolator 530. The linearly polarizedexcitation light after passing through optical isolator 530 is referredto as excitation light 420. PBS 540 may be configured to transmit lighthaving a vibration orientation parallel with that of excitation light420 and reflect light having a vibration orientation orthogonal to thatof excitation light 420. For example, excitation light 420 may behorizontally polarized or have a vibration orientation in a horizontaldirection. PBS 540 may transmit horizontally polarized light and reflectvertically polarized light. Therefore, excitation light 420 transmitsthrough PBS 540 and propagates towards SLM 320 a.

Description below of the polarization-based scheme of recycling system500 uses excitation light 420 that is horizontally polarized as anexample. Embodiments of the polarization-based scheme of recyclingsystem 500 is equally applicable for linearly polarized excitation light420 having any vibration orientation.

As described above, when SLM 320 a is a DMD, a small fraction ofexcitation light 420 may be reflected by the “on” micromirrors of theDMD towards lens 330 a along the optical axis, while the off-axisexcitation light 403 reflected by the “off” pixels are steered away fromthe optical axis. Mirror 510 a may be configured to intercept theoff-axis excitation light 403 and reflect it back to the “off” pixels onthe DMD. Off-axis excitation light 403 may pass through QWP 550 a firsttime when it propagates towards mirror 510 a and a second time when itis directed back to the DMD by mirror 510 a, which rotate the vibrationorientation of off-axis excitation light 403 by 90°. For example,horizontally polarized excitation light 403 may be changed to bevertically polarized after passing through QWP 550 twice. The verticallypolarized excitation light is then reflected by the “off” micromirrorsof the DMD towards PBS 540.

Because the vertically polarized excitation light reflected to PBS 540has a vibration orientation orthogonal to that of horizontally polarizedexcitation light 420, it is reflected by PBS 540 and directed to mirror510 b. Without changing its vibration orientation, mirror 510 b and PBS540 then reflect the vertically polarized excitation light back to theDMD, where the “on” micromirrors then reflect the vertically polarizedexcitation light towards lens 330 a along the optical axis. The “off”micromirrors reflect the vertically polarized excitation light, whichagain transmits through QWP 550 and mirror 510 a twice and becomeshorizontally polarized. This horizontally polarized excitation lightwould pass through PBS 540, but would not propagate back to light source310 because of optical isolator 530.

In the above-described polarization-based scheme of recycling system500, because QWP 550 rotates the vibration orientation of off-axisexcitation light 403 by 90°, excitation light 404 reflected towards theoptical axis, which includes the portion of the off-axis excitationlight 403 that is recycled, would have orthogonal polarizations. In suchinstances, rather than a polarizing beamsplitter, beamsplitter 350 maysuitably be a multiband dichroic that has multiple cut-off wavelengthsand passbands. As described above, the multiband dichroic may beselected such that wavelengths of excitation light 402 having a discretespectrum are substantially reflected and wavelengths of emittedfluorescent light 408 are substantially transmitted. Therefore, thispolarization-based scheme may work better in systems 300 using a lightsource 310 having discrete wavelengths or narrow spectra bands, such asa combination of a set of lasers operating at discrete wavelengths.

Confocal Optical Sectioning Capability

As described above, system 300 may allow for confocal opticalsectioning, which allows for selecting the depth of a focal plane in thesample. The depth of the focal plane may be selected by introducing oneor more optical pinholes at a plane conjugate to the selected focalplane.

FIG. 8 is a schematic representation of an exemplary system 300 thatallows for confocal optical sectioning. As shown in FIG. 8, system 300may include the first SLM 320 a for generating excitation pattern 100 asdescribed above, a second SLM 320 b for confocal optical sectioning, atleast one additional mirror 510 c, one or more tube lenses, e.g., 330 dand 330 e, and a z-axis translation stage or a tunable liquid lens (notshown). SLM 320 b may have similar types and features as described abovefor SLM 320 a. For example, pixels of SLM 320 b may be individuallymodulated in the same manners as those described for SLM 320 a.

SLM 320 b may be placed at about a plane conjugate to a focal planelocated at a desired depth in the sample along the optical axis. Forexample, lens 330 b and objective 360 may form an imaging configuration.As shown in FIG. 8, lens 330 b may be located behind objective 360 andSLM 320 b may be located about one focal length behind lens 330 b. Thespace between the back aperture of objective 360 and lens 330 b is acollimated space, which may be adjusted as need based on variousfactors, such as the geometry of system 300 and a desired location of aminimum beam aperture. In some embodiments, lens 330 b is placed aboutone focal length behind objective 360.

Pixels of SLM 320 b may be selectively actuated or switched to “on” or“off” states to form a pinhole pattern matching or conjugatingexcitation pattern 100 on the sample. The pinhole pattern may include aplurality of artificial optical pinholes at the conjugate plane andreject out-of-focus fluorescent light from the sample. Therefore,out-of-focus fluorescent light would not pass through the detectionsystem and are substantially removed or eliminated from the acquired 2-Dimage 200.

The size and separations of the artificial pinholes in the pinholepattern are programmable, and may be customized based on themagnification of the imaging configuration formed by objective 360 andlens 330 b. In some instances, the pinhole pattern may include aplurality of “on” pixels in elongated shapes to allow fluorescent lightemitted from multiple locations on the sample (e.g., areas excited byexcitation spots 112 a-112 f) to be acquired simultaneously. In otherinstances, the pinhole pattern may include an array of “on” pixels thatmatch the size of the excitation lines or excitation spots in excitationpattern 100.

The fluorescent light 412 reflected by the “on” pixels of SLM 320 b isthen imaged to imaging device 380 by tube lenses 330 d and 330 e. Forexample, mirror 510 c may be placed at a suitable position along theoptical axis and for directing fluorescent light 412 reflected by the“on” pixels to the tube lenses. Tube lens 330 d may be located about onefocal length beyond the image produced by lens 330 b (e.g., about onefocal length behind SLM 320 b) such that it re-collimates thefluorescent light from the sample. Imaging device 380 may be locatedabout one focal length behind tube lens 330 e or at a conjugate plane ofSLM 320 b. Because the fluorescent light is collimated in the spacebetween tube lenses 330 d and 330 e, the distance between tube lenses330 d and 330 e may be adjusted as desired. In some embodiments, tubelens 330 e may be about two focal lengths behind tube lens 330 d suchthat a plane midway between tube lens 330 d and 330 e is conjugate to anexit pupil of system 300.

By digitally changing and/or laterally shifting excitation pattern 100and the matching pinhole pattern on SLM 320 b correspondingly, the wholefield of view may be scanned for acquiring a confocal-imaging dataset.By further scanning the field of view across the sample, the wholesample can be scanned to obtain a complete confocal-imaging dataset ofthe sample.

In some embodiments, imaging device 380 may be suitably tilted to reduceaberrations and thus improve the quality of the acquired 2-D imagedataset. This is at least because the “on” pixels of SLM 320 b directfluorescent light 412 at an angle that is not perpendicular to thesurface plane of SLM 320 b such that an image plane formed by tubelenses 330 d and 330 e may be tilted. Aberrations caused by this tiltingeffect may be compensated by properly tilting imaging device 380.Aberrations may be further reduced if a dispersion angle of dispersiveelement 340 b is adjusted to be parallel to a rotation axis of thetilted imaging device 380.

To change or select a depth of the focal plane, in some embodiments,sample holder 370 may be installed on the z-axis translation stage. Thedesired depth of the focal plane may be selected by moving sample holder370 along the optical axis using the z-axis translation stage.Alternatively, objective 360 may be installed on the z-axis translationstage and the desired depth of the focal plane may be selected by movingobjective 360 along the optical axis. As describe herein, the z-axistranslation stage may also include x-y translation capability to movethe field of view of system 300 across the sample in lateral directions.In other embodiments, the desired depth of the focal plane may beselected by tuning the focus of a tunable liquid lens (not shown) placedbehind objective 360. Additionally, the z-translation stage or thetunable liquid lens may be controlled by a computer program to achieveautofocusing.

Advantageously, a degree of confocality may be adjusted as needed bychanging the size and/or separation of the artificial pinholes formed bySLM 320 b. For example, increasing the sizes of the pinholes byincreasing the number of pixels in the pinholes and/or reducing thepinhole spacing may reduce the degree of confocality and thus the degreeof depth selectivity of the desired focal plane. On the other hand,decreasing the size of the pinholes by reducing the number of pixels inthe pinholes and/or increasing the pinhole spacing may increase thedegree of confocality and thus the degree of depth selectivity of thedesired focal plane. In some embodiments, the depth selectivity may beproportional to the ratio of the number of “off” and “on” pixels of SLM320 b. Therefore, SLM 320 b may advantageously allow for switchingbetween wide-field and confocal imaging as desired by convenientlyadjusting the pinhole size and/or separation.

Additionally, the pinhole pattern formed by pixels of SLM 320 badvantageously allows for confocal imaging of a plurality of areas onthe sample simultaneously illuminated by excitation pattern 100. Thismay increase the speed and/or throughput of acquiringhyperspectral-imaging datasets across the sample at the desired focalplane comparing to traditional confocal microscopes that use sequentialpoint-by-point scanning.

As shown in FIG. 8, in embodiments of system 300 using SLM 320 b,dispersive element 340 b may be located in the collimated space betweentube lenses 330 d and 330 e. Because the pinhole pattern on SLM 320 bmatches excitation pattern 100 on the sample, fluorescent light 412reflected by the artificial pinholes of SLM 320 b can be dispersed bydispersive element 340 b as described above such that the fluorescenceemission spectra corresponding to the excitation spots of excitationpattern 100 can be acquired by the 2-D sensor of imaging device 380.

Selective Filtering of Fluorescence Emission Spectrum

In some applications, different fluorophores having fluorescenceemission spectra that are spaced apart, such as green and redfluorophores, may be used or exist in the sample. This may result inlateral gaps in a fluorescence emission spectrum acquired in 2-D image200 along the emission wavelength axis, resulting in inefficient use ofthe space on the 2-D sensor of imaging device 380.

In other applications, the combination of different fluorophores mayresult in an overall broad fluorescence emission spectrum to be acquiredby imaging device 380. In some instances, multiple spectral regionswithin the broad emission fluorescence spectrum may be more useful thanother regions. Acquiring the complete broad fluorescence emissionspectrum may result in inefficient use of the space on the 2-D sensor ofimaging device 380 and further reduce the throughput of acquiring thehyperspectral-imaging dataset.

To increase the efficiency of using the sensor space of imaging device380 and increase the throughput of system 300, a spectral slicing system342 may be included at a collimated space along the optical axis in thedetection system. For example, as shown in FIG. 8, spectral slicingsystem 342 may be located between tube lenses 330 d and 330 e, and maybe placed before dispersive element 340 b in the detection system.Spectral slicing system 342 may selectively pass one or more spectralbands with tunable bandwidths and/or center wavelengths, therebyallowing for acquiring a hyperspectral-imaging dataset with desiredspectral bands and/or desired spectral resolutions.

As shown in FIG. 8, spectral slicing system 342 may include a pluralityof spectral slicing modules 344. Fluorescent light 408 emitted by thesample may enter spectral slicing system 342 after being collimated orre-collimated. Using one or more beamsplitters and/or motorized flipmirrors, spectral slicing system 342 may split an input collimatedfluorescent light beam into one or more beams, each having a differentspectral band, and direct them through spectral slicing modules 344respectively. Each spectral slicing module 344 may filter one of thebeams to have a desired bandwidth and a center wavelength. After thefiltering, spectral slicing system 342 may combine the filtered beamsinto an output beam using one or more beamsplitters and/or motorizedflip mirrors.

Spectral slicing modules 344 may each operate as a tunable bandpassfilter with a tunable passband width and/or a tunable center wavelength.For example, spectral slicing module 344 may include a long-pass filterand a short-pass filter along its optical axis. At least one of thelong-pass filter and short-pass filter is rotatable relative to theoptical axis. Rotating the filters may adjust the angle of incidence ofthe beam on the filters and thus shift the wavelengths of theirabsorption or reflection edges. Thus, rotating the long-pass filterand/or short-pass may tune the bandwidth and/or center wavelength of thespectral passband formed by the long-pass and shot-pass filters.Alternatively, spectral slicing modules 344 may each include a tunablebandpass filter whose passband may be tuned by rotating the filter andthus tuning the angle of incidence of the beam on the filter.

Spectral slicing system 342 allows the measured fluorescence emissionspectra to be adjustably filtered to desired spectral ranges useful fora particular application. By selecting the desired spectral ranges, thespace on the 2-D sensor of imaging device 380 can be used moreefficiently. For example, as described above, the degree of dispersioncaused by dispersive element 340 b can be adjustable. The spectralresolution of the selected spectral ranges of the fluorescence emissionspectra may be increased by increasing the degree of spectral dispersionusing dispersive element 340 b, thereby providing more information ofthe fluorophores or fluorescent molecules in the sample.

Additionally, selecting the desired spectral ranges may allow forreducing the lateral spacing between the fluorescence emission spectrain 2-D image 200 along the emission wavelength axis, thereby improvingthe throughput of dataset acquisition. For example, by reducing theperiod of excitation pattern 100 in the horizontal direction, anddecreasing the degree of spectral dispersion using dispersive element340 b, the period of the array of fluorescence emission spectra in thehorizontal direction in 2-D image 200 may be reduced. This may in turnincrease the number of fluorescence emission spectra that can beacquired in one exposure, thereby increasing the efficiency of using thesenor space of imaging device 380.

Alternative Configurations

In some applications, more compact configurations of system 300 may bedesirable. In such instances, system 300 may use diffractive elements inplace of SLM 320 a and/or SLM 320 b. Embodiments of such configurationsof system 300 are described below in reference to FIGS. 9-13.

FIG. 9 is a schematic representation of an exemplary compact embodimentof system 300. As shown in FIG. 9, system 300 may advantageously use atransmission-type illumination to simplify its geometry. However,reflection-type illumination configurations as shown in FIGS. 3-8 mayalso be used depending on the application. In the illumination system,excitation light 402 from light source 310, such as a supercontinuumlaser source provided through an optic fiber, is collimated by lens 330a, transmits through a first diffractive element 600 a, and thenilluminates a sample placed on sample holder 370. Diffractive element600 a modulates the phase of excitation light 402 transmitting throughit and structures excitation light 402 for generating excitation pattern100. The phase modulation may render a plurality of wavelets of thetransmitted excitation light 430 with different directions and/orphases, generating a diffraction pattern in the far field. Thediffraction pattern focused on the sample is referred to as excitationpattern 100.

FIG. 12 is a schematic representation of an exemplary diffractiveelement 600 a. As shown in FIG. 12, in some embodiments, diffractiveelement 600 a may be a 2-D array of diffractive lenses 610. Forexcitation light 402 having a single wavelength, diffractive element 600a generates a 2-D array of excitation spots, one by each diffractivelens 610. For excitation light 402 having multiple discrete wavelengthsor a range of wavelengths, different wavelengths of excitation light 402are diffracted by each diffractive lens 610 into several beamstravelling in different angular directions. Therefore, when focused onthe sample, the different wavelengths of excitation light 402 may havefocuses spatially shifted from one another in a first lateral direction(e.g., vertical direction), thereby generating excitation pattern 100 asshown in FIG. 1 or FIG. 2.

In some embodiments, diffractive lenses 610 of diffractive element 600 amay be zone plates that have transparent and nontransparent bands,conventional gratings made by, e.g., binary lithography, grayscalelithography, or molding processes, or subwavelength gratings made bybinary lithography. In other embodiments, diffractive element 600 a maybe replaced with a 2-D lenslet array and a transmissive diffractiongrating that have the phase modulation capability for generatingexcitation pattern 100 as described above.

In the detection system, fluorescent light 408 emitted by the sample iscollected and collimated by objective 360, transmits through dispersiveelement 340 b, and is then focused onto imaging device 380 by lens 330b. Dispersive element 340 b may spectrally disperse fluorescence light408 in a second lateral direction (e.g., horizontal direction) asdescribed above. Dispersive element 340 b may have the same features andfunctions as described above.

In some embodiments, system 300 may include a second linear polarizer390 c. Fluorescent light 408 may pass through polarizer 390 c. Whenexcitation light 402 is linearly polarized, polarizer 390 c may be usedto substantially reflect the polarized excitation light and thus blockit from reaching imaging device 380. In other embodiments, a set ofnotch filters or a single multi-notch filter (not shown) may be added tothe detection system along the optical axis.

Because diffractive element 600 a does not have the digitalprogrammability as that of an SLM, either diffractive element 600 a orsample holder 370 may be translated in spatial dimensions to scanexcitation pattern 100 across the field of view or the sample to obtaina complete 4-D hyperspectral-imaging dataset. The scanning scheme may bethe same as described above in reference to FIGS. 1 and 2. Differentareas in each scanning cell 110 may be illuminated by spatially shiftingexcitation pattern 100 in the vertical and horizontal directions. Ateach spatial position of excitation pattern 100, at least one 2-D image200 of fluorescence emission spectra of the illuminated areas can beacquired. Then, a plurality of 2-D images 200 of fluorescence emissionspectra can be acquired corresponding to a series of excitation patterns100 laterally shifted from one another and used for reconstructing the4-D hyperspectral-imaging dataset.

FIG. 10 is a schematic representation of another exemplary compactembodiment of system 300. System 300 as shown in FIG. 10 may allow foracquiring a 4-D hyperspectral-imaging dataset by performing scanning inone lateral direction. For example, system 300 may include diffractiveelement 600 a in the detection system and another diffractive element600 b in the illumination system. As shown in FIG. 10, in theillumination system, excitation light 402 from light source 310 iscollimated by lens 330 a, transmits through diffractive element 600 b,and then illuminates a sample placed on sample holder 370.

FIG. 13 is a schematic representation of an exemplary diffractiveelement 600 b. Diffractive element 600 b may modulate the phase ofexcitation light 402 transmitting through it and render wavelets oftransmitted excitation light 430 having different directions and/orphases. In some embodiments, diffractive element 600 b may include alinear array of diffractive cylindrical lenslets 620. For excitationlight 402 of a single wavelength, diffractive element 600 b generates arepeating pattern of single-colored stripes, one by each cylindricallenslet 620. For excitation light 402 having multiple discretewavelengths or a range of wavelengths, different wavelengths ofexcitation light 402 are diffracted by each cylindrical lenslet 620 intoseveral beams travelling in different angular directions. Therefore,when focused on the sample, different wavelengths of excitation light402 may have focuses spatially shifted from one another in a firstlateral direction (e.g., vertical direction), generating a repeatingpattern of a series of shifted different-colored stripes. Depending onthe spectrum of light source 310, the different-colored stripes may beconnected or separated in the first lateral direction. The repeatingpattern of shifted different-colored stripes are then illuminated on thesample.

In the detection system, rather than using dispersive element 340 b,diffractive element 600 a may be added and placed in front of imagingdevice 380. Fluorescent light 408 emitted by the sample is collected andcollimated by objective 360, transmits through polarizer 390 c, and isthen imaged onto diffractive element 600 a by lens 330 b. Diffractivelenses 610 of diffractive element 600 a may then spectrally disperse thefluorescent light in a second lateral direction (e.g., horizontaldirection) and image the spectrally dispersed fluorescent light 410 tothe 2-D sensor of imaging device 380.

In some embodiments, the focal length of lens 330 b is selected suchthat a diffraction-limited spot size of lens 330 b at its focal planemay cover a plurality of pixels of the 2-D sensor of imaging device 380.This may affect the numerical aperture (NA), the focal ratio (f-ratio),and/or the magnification of lens 330 b. For example, to increase thediffraction-limited spot size of lens 330 b, lens 330 b may have alonger focal length, a smaller NA or a larger f-ratio, and/or a greatermagnification.

Diffractive element 600 a may be designed or selected such that thediameters of its diffractive lenses 610 is about the size of adiffraction-limited spot of lens 330 b. Different wavelengths of thefluorescent light 410 deflected and focused by each diffractive lens 610may have focuses spatially shifted from one another in the secondlateral direction, generating an array of fluorescence emission spectraas shown in FIG. 1 or FIG. 2.

Embodiments of system 300 as shown in FIG. 10 allows for acquiringfluorescence emission spectra in a 2-D image 200 as shown in FIG. 1 orFIG. 2 for areas or locations on the sample illuminated by the repeatingpattern of a series of laterally shifted different-colored stripes. Toacquire the excitation spectra, the repeating pattern may be scannedalong the first lateral direction such that the areas or locations onthe sample previously illuminated by a colored stripe of the repeatingpattern are illuminated by a different-colored stripe. The shifting ofthe repeating pattern in the first lateral direction and subsequentacquisition of a corresponding 2-D image 200 may be performed for aplurality times. In such instances, fluorescence emission spectracorresponding to the excitation wavelengths for each area or location onthe sample can be acquired and used for reconstructing the 4-Dhyperspectral-imaging dataset.

In the embodiments of system 300 as shown in FIG. 10, because therepeating pattern of a series of laterally shifted different-coloredstripes is continuous in the second lateral direction, the repeatingpattern may only need to be scanned along the first lateral direction toobtain the excitation spectra for all areas or locations within thefield of view. This may further improve the throughput and efficiency ofsystem 300 for acquiring the 4-D hyperspectral-imaging dataset.

Along the second lateral direction, each area illuminated by thecontinuous colored stripes can be imaged to a diffractive lens 610,which then disperses the fluorescent light and focuses it to imagingdevice 380. In such instances, the spatial resolution along the secondlateral direction may depend on the size and focal length of diffractivelenses 610, the focal lengths of lens 330 b and objective 360, and/orthe size of the 2-D sensor of imaging device 380. In some embodiments,increasing the focal length of lens 330 b may allow for using largerdiffractive lenses 610. The spectral resolution along the second lateraldirection may depend on the width and/or focal length of diffractivelenses 610, and the off-axis focal shifts generated by diffractivelenses 610 in the second lateral direction. For example, increasinggroove density of diffractive lenses 610 would increase the diffractionangles of the fluorescent light and thus the off-axis focal shifts,thereby increasing the spectral resolution in the second lateraldirection.

FIG. 11 is a schematic representation of another exemplary compactembodiment of system 300 that provides the capability for measuringfluorescence polarization. As shown in FIG. 11, system 300 may includetwo polarizers 390 a and 390 c. Polarizer 390 a may be at a suitableplace along the optical axis in the illumination system, therebygenerating linearly polarized excitation light. Polarizer 390 c may beat a suitable place along the optical axis in the detection system,thereby transmitting emitted fluorescent light 408 having a givenvibration orientation. To perform fluorescence polarization assays, thetransmission axis of polarizer 390 c may be rotated between orientationsparallel and orthogonal to the vibration orientation of the linearlypolarized excitation light. 2-D images 200 of the fluorescence emissionspectra of fluorescent light 408 having vibration orientations parallelto and orthogonal to that of the polarized excitation light may berespectively acquired by imaging device 380. The acquired 2-D images 200may then be used for fluorescence polarization (or anisotropy) assay.

System 300 as described herein may be utilized in a variety of methodsfor hyperspectral imaging. FIG. 14 is a flowchart of an exemplary method700 for performing hyperspectral imaging or for acquiring ahyperspectral-imaging dataset of a sample. Method 700 uses system 300and features of the embodiments of system 300 described above inreference to FIGS. 3-13.

At step 702, light source 310 having a discrete spectrum or a continuousspectrum is provided and configured to emit excitation light 402 havingone or more wavelengths. At step 704, excitation light 402 is structuredby SLM 320 a to into a predetermined two-dimensional pattern at aconjugate plane of a focal plane in the sample. At step 706, thestructured excitation light, e.g., excitation light 404 reflected by SLM320 a, is spectrally dispersed by dispersive element 340 a in a firstlateral direction. At step 708, spectrally dispersed excitation light406 is directed towards and focused on the sample, illuminating thesample in excitation pattern 100 with the one or more wavelengthsdispersed in the first lateral direction. At step 710, fluorescent light408 collected from the sample is spectrally dispersed by dispersiveelement 340 b in a second lateral direction. At step 712, spectrallydispersed fluorescent light 410 is imaged to a 2-D sensor of imagingdevice 380.

Method 700 may further include additional steps. For example, method 700may include calibrating system 300 before acquiring 2-D image 200.Various optical components in system 300 may be suitably calibrated andaligned such that focused 2-D images 200 with reduced or minimumdistortion can be acquired.

Method 700 may further include polarizing excitation light 402 to bedirected to the sample using a first polarizer, and substantiallyreflecting light collected from the sample having the same polarizationas that of the polarized excitation light using a second polarizer or apolarizing beamsplitter (PBS).

Method 700 may further include illuminating the sample sequentially in aseries of excitation patterns 100 laterally shifted from one another,and obtaining a plurality of 2-D images 200 of the spectrally dispersedemission light corresponding to the series of excitation patterns 100,and reconstructing the plurality of 2-D images 200 to provide a 4-Dhyperspectral-imaging dataset. As described above, each 2-D image 200records an array of fluorescence emission spectra corresponding to eachlaterally shifted excitation pattern 100.

Method 700 may further include providing programmable artificial opticalpinholes at a plane conjugate to the focal plane by SLM 320 b, forming aseries of pinhole patterns by pixels of SLM 320 b, and matching theseries of pinhole patterns to the series of excitation patterns 100. Asdescribed above, light collected from SLM 320 b is imaged to imagingdevice 380 using one or more lenses. A 2-D image 200 of the spectrallydispersed emission light may be acquired after each lateral shift ofexcitation pattern 100 and the formation of its matching pinholepattern. Method 700 may further include reconstructing the 2-D images200 corresponding to the series of excitation patterns 100 to provide a4-D hyperspectral-imaging dataset of the selected focal plane of thesample.

The foregoing description has been presented for purposes ofillustration. It is not exhaustive and is not limited to precise formsor embodiments disclosed. Modifications and adaptations of theembodiments will be apparent from consideration of the specification andpractice of the disclosed embodiments. For example, the describedimplementations include hardware and software, but systems and methodsconsistent with the present disclosure can be implemented as hardwarealone. In addition, while certain components have been described asbeing coupled to one another, such components may be integrated with oneanother or distributed in any suitable fashion.

Moreover, while illustrative embodiments have been described herein, thescope includes any and all embodiments having equivalent elements,modifications, omissions, combinations (e.g., of aspects across variousembodiments), adaptations and/or alterations based on the presentdisclosure. The elements in the claims are to be interpreted broadlybased on the language employed in the claims and not limited to examplesdescribed in the present specification or during the prosecution of theapplication, which examples are to be construed as nonexclusive.Further, the steps of the disclosed methods can be modified in anymanner, including reordering steps and/or inserting or deleting steps.

Instructions or operational steps stored by a computer-readable mediummay be in the form of computer programs, program modules, or codes. Asdescribed herein, computer programs, program modules, and code based onthe written description of this specification, such as those used by thecontroller, are readily within the purview of a software developer. Thecomputer programs, program modules, or code can be created using avariety of programming techniques. For example, they can be designed inor by means of Java, C, C++, assembly language, or any such programminglanguages. One or more of such programs, modules, or code can beintegrated into a device system or existing communications software. Theprograms, modules, or code can also be implemented or replicated asfirmware or circuit logic.

The features and advantages of the disclosure are apparent from thedetailed specification, and thus, it is intended that the appendedclaims cover all systems and methods falling within the true spirit andscope of the disclosure. As used herein, the indefinite articles “a” and“an” mean “one or more.” Similarly, the use of a plural term does notnecessarily denote a plurality unless it is unambiguous in the givencontext. Words such as “and” or “or” mean “and/or” unless specificallydirected otherwise. Further, since numerous modifications and variationswill readily occur from studying the present disclosure, it is notdesired to limit the disclosure to the exact construction and operationillustrated and described, and accordingly, all suitable modificationsand equivalents may be resorted to, falling within the scope of thedisclosure.

Other embodiments will be apparent from consideration of thespecification and practice of the embodiments disclosed herein. It isintended that the specification and examples be considered as exampleonly, with a true scope and spirit of the disclosed embodiments beingindicated by the following claims.

What is claimed is:
 1. A hyperspectral imaging system, comprising: asample holder configured to hold a sample; a light source configured toemit excitation light having one or more wavelengths; a two-dimensional(2-D) imaging device; and an optical system comprising: a first spatiallight modulator (SLM); a first dispersive element; and a seconddispersive element; wherein the optical system is configured to (i)structure the excitation light, using the first SLM, into apredetermined two-dimensional excitation pattern at a conjugate plane ofa focal plane in the sample; (ii) spectrally disperse, using the firstdispersive element, the excitation light; (iii) use excitation patternto illuminate the sample illuminate the sample in an excitation patternwith the one or more wavelengths dispersed along a first lateraldirection within the focal plane; (iv) collect, from the sample,emission light; (v) spectrally disperse, using the second dispersiveelement, the collected emission light; and (vi) provide the collectedemission light in-focus to the imaging device such that the spectrallydispersed emission light is spectrally dispersed along a second lateraldirection.
 2. The system of claim 1, wherein the optical system furthercomprises an objective, wherein the objective focuses the excitationlight used to illuminate the sample and collimates the emission lightcollected from the sample.
 3. The system of claim 1, wherein the opticalsystem further comprises an optical beamsplitter, wherein the opticalbeamsplitter directs the excitation light onto the sample and directsthe emission light collected from the sample to the imaging device. 4.The system of claim 1, wherein the optical system further comprises afirst polarizer configured to polarize the excitation light.
 5. Thesystem of claim 4, wherein the optical system further comprises at leastone of a second polarizer or a polarizing beamsplitter (PBS) configuredto reflect light having the same polarization as that of the polarizedexcitation light.
 6. The system of claim 1, wherein the first SLMcomprises at least one of a digital micromirror device (DMD), adiffractive element, a liquid crystal device (LCD), or a liquidcrystal-on-silicon (LCOS) device.
 7. The system of claim 1, furthercomprising an excitation light-recycling system, wherein a selection ofpixels of the first SLM directs a portion of the excitation light awayfrom an optical axis of the hyperspectral imaging system, and whereinthe excitation light-recycling system recycles the portion of theexcitation light that is directed away from the optical axis of thehyperspectral imaging system by the SLM.
 8. The system of claim 7,wherein the excitation light-recycling system comprises at least onelens and at least one mirror, wherein the at least one lens and the atleast one mirror of the excitation light-recycling system direct andcollimate the excitation light directed away from the optical axis ofthe hyperspectral imaging system back to the optical axis of thehyperspectral imaging system via one or more reflections.
 9. The systemof claim 7, wherein the excitation light-recycling system comprises aPBS, at least two plane mirrors, and a quarter-wave plate, wherein thePBS, at least two plane mirrors, and quarter-wave plate of theexcitation light-recycling system direct the excitation light directedaway from the optical axis of the hyperspectral imaging system back tothe optical axis of the hyperspectral imaging system via one or morereflections.
 10. The system of claim 1, wherein the optical systemfurther comprises a second SLM, wherein pixels of the second SLM operateas programmable artificial optical pinholes to form a pinhole patternthat matches the excitation pattern at the plane conjugate to the focalplane, and wherein the optical system provides collected emission lightfrom the focal plane in-focus to the second SLM.
 11. The system of claim1, wherein at least one of the first dispersive element or the seconddispersive element comprises a diffraction grating or a prism.
 12. Thesystem of claim 1, wherein the light source comprises at least one of ahigh-pressure mercury lamp, a xenon lamp, a halogen lamp, a metal halidelamp, a supercontinuum light source, a laser, or an LED.
 13. The systemof claim 1, further comprising a controller operably coupled to thefirst SLM, the light source, and the imaging device, wherein thecontroller is configured to operate the first SLM to modulate pixels ofthe first SLM, to operate the light source, and to operate the imagingdevice.
 14. The system of claim 1, wherein the excitation patterncomprises an array of excitation spots or excitation stripes.
 15. Amethod for hyperspectral imaging, comprising: providing, from a lightsource, excitation light with one or more wavelengths; structuring, by afirst spatial light modulator (SLM), the excitation light from the lightsource into a predetermined two-dimensional excitation pattern at aconjugate plane of a focal plane in a sample; spectrally dispersing, bya first dispersive element, the excitation light; using the excitationpattern to illuminate the sample with the one or more wavelengthsdispersed along a first lateral direction within the focal plane;spectrally dispersing, by a second dispersive element, emission lightcollected from the sample; and imaging, using a two-dimensional (2-D)imaging device, the spectrally dispersed emission light, wherein thespectrally dispersed emission light is received in-focus by the imagingdevice such that the spectrally dispersed emission light is spectrallydispersed along a second lateral direction.
 16. The method of claim 15,further comprising polarizing, by a first polarizer, the excitationlight; and reflecting, by at least one of a second polarizer or apolarizing beamsplitter (PBS), light having the same polarization asthat of the polarized excitation light.
 17. The method of claim 15,wherein the first SLM comprises at least one of a digital micromirrordevice (DMD), a diffractive element, a liquid crystal device (LCD), or aliquid crystal-on-silicon (LCOS) device.
 18. The method of claim 15,wherein structuring, by the first spatial light modulator (SLM), theexcitation light from the light source comprises directing a portion ofthe excitation light away from an optical axis of the hyperspectralimaging system, wherein the method further comprises: recycling theportion of the excitation light that is directed away from the opticalaxis of the hyperspectral imaging system.
 19. The method of claim 15,further comprising illuminating the sample sequentially in a series ofexcitation patterns that are laterally shifted, within the focal planewithin the sample, from each another; obtaining, using the imagingdevice, a plurality of 2-D images of the spectrally dispersed emissionlight, wherein each of the obtained 2-D images corresponds to arespective one of the excitation patterns in the series of excitationpatterns; and reconstructing the obtained plurality of 2-D images toprovide a 4-D hyperspectral-imaging dataset.
 20. The method of claim 19,further comprising providing, by a second SLM, a series of patterns ofartificial optical pinholes at a further plane that is conjugate to thefocal plane, wherein each pattern of artificial optical pinholes in theseries of patterns of artificial optical pinholes matches acorresponding excitation pattern in the series of excitation patterns.